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ABSTRACT 


This 


paper  describes  the  evolution  of  the 


between  methods  and  applications  continues 
today  in  an  environment  which  we  now  briefly 
describe. 


Riak  and  Decision  Analysis  Project  at  Lockheed- 
Geor g la which  develops  new  methods,  automates 
existing  methods,  and  performs  analyses  in 
support  of  program  definition  and  conceptual 
design  ofLjjew^alr  vehicles.  The  analytical 
method  a ''th  at  we  hav-  'found  most  useful  are 
identified.  Application  such  as  technology 
assessment  and  program  risk  analysis  are  dis- 
cussed. Observations  are  given  on  the  practice 
of  decision  analysis  in  an  engineering  environ- 
ment. 

INTRODUCTION 


Engineering  Systems  Analysis  Division  is 
one  of  two  divisions  controlled  by  the  Chief 
Advanced  Design  Engineer,  the  other  being 
Advanced  Design  Division.  Systems  Analysis  is 
responsible  for  requirements  definition  and  for 
performance  of  cost  and  effectiveness  evalua- 
tions of  proposed  designs.  Advanced  Design  is 
responsible  for  conceptual  and  preliminary 
design  studies,  for  both  new  and  derivative 
aircraft.  These  two  divisions  are  managed  by  a 
matrix  arrangement:  Each  working  level  engi- 

neer is  assigned  to  an  R&D  activity  (e.g.,  cost 
methods)  and  participates  in  one  or  more  inter- 
disciplinary projects  as  needs  arise. 


Interest  in  decision  analysis  at  Lockheed- 
Georgia  began  as  a result  of  a U.  S.  Air  Force 
(USAF)  competition.  On  15  October  1980  Loek- 
heed -Georgia,  along  with  Boeing  and  McDonuell- 
Douglas,  received  a Request  for  Proposal  (RFP) 
from  the  Air  Force  (Reference  1)  to  develop  and 
build  a transport  aircraft,  known  as  the  C-X 
(Cargo  Experimental).  The  C-X  was  specified  to 
be  a dual-purpose  airlifter,  having  both  inter- 
continental range  and  short  field  landing  snd 
takeoff  capabilities.  Responses  were  due  on  15 
January  1981. 

Oie  of  the  proposal  volutes  specified  by  the 
FFP  was  entitled  simply  "Risk."  Lockheed- 
Georgia'a  Engineering  Systems  Analysis  Division 
(Figure  1)  waa  aaslgned  to  select  a atudy 
approach,  collect  information,  conduct  a risk 
analysis,  and  write  a 40  page  volume.  Unlike 
most  proposal  volutes  tdiich  describe  what  the 
contractor  will  do  if  he  is  selected,  the  Riak 
volute  waa  specified  to  be  a report  of  the 
reaulta  of  a thorough,  systematic  analysis. 
(>>e  of  the  authors  of  this  paper  waa  assigned 
as  senior  author  of  the  Risk  volute. 

Near  the  end  of  the  competition,  it  was 
recommended  to  management  that  when  the 
proposal  team  disbanded  and  Systems  Analysis 
reverted  to  requirements  analysis  and  methods 
development,  an  RAD  project  be  established  to 
continue  the  development  of  methods  for  risk 
and  decision  analysis.  (Xir  recommendation  was 
followed  and  the  Risk  and  Decision  Analysis 
Methodology  project  waa  established  with 
funding  for  one-half  man-year  of  methods  work. 
The  rest  of  our  funding  was  to  come  from  the 
projects  we  supported.  This  division  of  effort 


PROJECT  OBJECTIVES 

The  long-range  objectives  of  the  Risk  and 
Decision  Analysis  Methodology  project  are 
to  develop  a comprehensive  framework  of 
decision  science  methods,  and  implement  these 
methods  at  Lockheed-Georgia  to  assist  planners 
and  decisionmakers.  Annual  objectives  are  set 
to  enable  these  two  objectives  to  be  attained. 
These  short-range  objectives  specify  products 
(software  or  study  reports)  to  be  produced  by 
year's  end.  Careful  assessment  is  made  to 
assure  adequate  manpower  is  held  in  reserve  for 
applications  assignments , both  planned  and 
unplanned. 

The  term  "decision  science"  is  used  here  to 
mean  both  a problem-solving  philosophy  and  a 
collection  of  methods.  The  philosophy  is  to 
use  the  objectives  and  goala  of  the  manager  in 
structuring  the  decision  situation,  to  allow  as 
input  to  the  model  the  Judgment  of  the  manager 
and  his  staff,  and  to  give  explicit  considera- 
tion to  the  problem  environment  (timing, 
organizational  factors,  uncertainty,  and  con- 
straints). By  decision  science  methoda,  we 
mean  a collection  of  techniques  which  include, 
but  are  not  limited  to: 

1.  decision  analysis 

a.  decision  trees 

b.  multi attribute  utility  theory 

c.  probability  encoding 

d.  multiple  criteria  decision  models 

e.  decision  making  under  competition 

f.  policy  testing  via  System  Dynamics 


24 


• DATA  BASE  MANAGEMENT 

• COMPUTER  GRAPHICS 

• NUMERICAL  OPTIMIZATION 

• DECISION  SCIENCES 


Figure  1 . Immediate  Project  Environment 


2.  . risk  analysis 

s.  Monte  Carlo  simulation 
b.  iso-risk  contours 

o.  diagram  methods  for  potential  prob- 
lem identification 

3*  statistical  analysis 
network  analysis 

a.  PERT/CPM 

b.  CPM  Crashing 
o.  VERT 

d.  CERT 

The  first  three  of  these  a^e  the  methodology 
focus  of  the  1983  project,  as  shown  in  Figure 
2.  Next  year , a new  RAD  project  devoted  to 
network  methods  is  planned.  Figure  2 also 
illustrates  how  methodology  development  and 
applications  feed  each  other.  To  develop  our 
methods,  the  project  analysts  utilixe  the  open 
literature  and  intexact  with  theoreticians  at 


universities  and  government-sponsored  labora- 
tories. The  process  represented  is  the  well- 
known  "applied  research  circuit"  whereby 
theoretical  developments  eventually  find  their 
way  into  applications. 

BASIC  TOOLS  AND  EXAMPLE  APPLICATIONS 

There  are  certain  decision  analytic  methods 
which  we  have  used  repeatedly  over  the  last  3 
years.  Not  surprisingly,  they  are  not  the  more 
advanced  methods  available.  However,  these 
methods,  and  modifications/  combinations  of 
them,  provide  a well-rounded  decision  analysis 
capability  for  an  engineering  environment. 

The  methods  are: 

1.  Probability  Encoding 

2.  Monte  Carlo  Simulation 

3.  Multiattribute  Utility  Analysis 

«.  Critical  Path  Method 
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Probability  Encoding 

Because  most  variables  in  engineering 
studies  are  continuous  rather  than  discrete, 
the  decision  analyst  must  have  a method  for 
encoding  subjective  probability  distributions 
for  such  variables.  By  subjective  probability, 
we  mean  the  "Bayesian  approach"  which  inter- 
prets a distribution  as  one's  degree  of  belief 
about  the  outcome  of  future  events. 

We  use  a four-part  questionnaire  (Figure  3) 
to  convert  responses  from  a specialist  (techni- 
cal or  management)  into  a beta  distribution. 
In  the  example  shown,  we  have  asked  a relia- 
bility engineer  to  estimate  "Effective  Mission 
Capable  (EMC)  Rate"  for  an  aircraft  conceptual 
design.  In  giving  us  his  estimate,  he  takes 
into  account  all  reliability  analyses  conducted 
on  the  design,  together  with  his  experience  on 
previous  aircraft  development  programs.  In 
effect,  the  engineer  serves  as  the  data  base 
upon  which  the  estimate  of  probability  is 
based — he  interprets  the  uncertainty  much  in 
the  same  way  as  the  sample  mean  and  variance 


calculated  in  frequency-based  statistics. 
Caution  must  be  applied  in  extracting  a sub- 
jective probability  distribution:  assumptions 
upon  which  the  distribution  is  conditioned  must 
be  specified;  the  bias  in  using  a single  engi- 
neer to  provide  the  estimate  must  be  addressed 
(perhaps  through  the  use  of  a Delphi  approach). 

In  Figure  *4,  four  beta  distributions  are 
displayed.  Each  distribution  was  obtained  from 
the  project  engineer  responsible  for  the  varia- 
ble shown.  The  specific  values  for  end-points 
and  mode  were  removed  for  proprietary  reasons. 
The  information  displayed  is  valuable  in  it- 
self, but  the  real  payoff  is  in  using  these 
data  to  calculate  uncertainty  in  aircraft  range 
with  a given  mission  payload.  The  next  sub- 
section explains  how  this  is  accomplished. 

Monte  Carlo  Simulation 

We  use  Monte  Carlo  simulation  to  generate  a 
sample  distribution  on  an  output  variable  whose 
distribution  is  not  known  by  using  a functional 
relationship  between  this  output  variable  and 
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FOR  THE  VARIABLE  EMC  RATE,  PLEASE  ANSWER  THE  FOLLOWING: 


A.  LOWEST  VALUE  = 0.55  (LESSER  VALUES  HAVE  A NEGLIGIBLY  SMALL  PROBABILITY  OF  OCCURRING) 

B.  MOST  LIKELY  VALUE  = 0.66 

C.  HIGHEST  VALUE  =0.74  (GREATER  VALUES  HAVE  A NEGLIGIBLY  SMALL  PROBABILITY  OF  OCCURRING) 
0.  CONFIDENCE  IN  THE  MOST  LIKELY  VALUE  (CIRCLE  ONE) 
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Figure  3.  Conversion  of  Questionnaire  Responses  to  Beta 
Distribution 
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Figure  4.  Probability  Distributions  for  Variables  Upon  which 
Range  is  Dependent 


input  variables  with  subjectively  estimated 
distributions.  The  process,  also  known  as 
"quantitative  uncertainty  analysis,"  is  shown 
in  Figure  5.  In  some  cases,  we  calibrate  the 
function  with  the  results  of  prior,  determinis- 
tic analyses  by  means  of  either  an  additive  or 
a multiplicative  constant.  For  instance,  with 
the  aircraft  range  example,  we  substituted  the 
nominal  values  of  the  input  variables  into  the 
Breguet  range  equation  and  solved  for  a multi- 
plicative calibration  constant  that  would  yield 
the  nominal  value  for  range.  Once  the  function 
is  calibrated,  Monte  Carlo  simulation  quickly 
develops  a sample  distribution  for  the  output 
variable.  A sample  mean  and  variance  may  be 
calculated,  and  a theoretical  distribution  may 
be  fitted.  Figure  6 shows  the  results  from  the 
aircraft  range  example.  The  probabilities 
shown  are  the  kind  of  information  management 
needs.  Note  that  no  one  individual  on  the 
project  could  have  answered  the  question, 
"What's  the  uncertainty  in  this  aircraft's 
range?"  The  value  of  the  distribution  on  range 
is  highlighted  when  comparisons  among  aircraft 
have  to  be  made. 

(Xiantitative  uncertainty  analysis  need  not 
be  restri^ed  «-o  a single  equation,  such  as  in 
the  range  example.  Often,  an  entire  model 


(systems  of  equations)  has  been  linked  as  a 
subprogram  to  the  main  Monte  Carlo  simulation 
program.  An  example  would  be  linking  with  the 
USAF  Cost  Oriented  Resources  Estimating  (CORE) 
model  to  quantify  uncertainty  in  airlifter 
fleet  0 & S cost.  Furthermore,  at  Lnckheed- 
Georgia  uncertainty  analysis  processes  are 
being  performed  sequentially  to  conduct  what 
may  be  termed  "large-scale  uncertainty  analy- 
sis." To  illustrate,  consider  Figure  7 which 
depicts  the  data  flow  for  the  Airlift  Fleet 
Cost-Effectiveness  Uncertainty  Estimator 
(AFCUE)  model  (Reference  2). 

In  AFCUE,  uncertainty  analysis  is  repeatedly 
used  to  convert  distributions  on  independent 
variables  into  distributions  on  dependent 
variables.  The  objective  is  to  convert  un- 
certainty in  key  technical  aircraft  variables 
into  uncertainty  in  fleet  life  cycle  cost.  By 
performing  this  process  on  aircraft  with  dif- 
ferent technology  mixes,  the  effects  of  tech- 
nological uncertainty  on  typical  "point-value" 
estimates  in  conceptual  design  may  be  judged. 
Advanced  technologies,  while  offering  signifi- 
cant performance  benefits,  also  introduce  risk 
into  all  estimates  of  performance,  effective- 
ness, and  cost.  AFCUE  quantifies  this  risk,  in 


Figure  5.  Flow  Diagram  far  Quantitative  Uncertainty  Analysis 
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effect  layering  an  uncertainty  study  over  the 
standard  conceptual  design  process.  This  seems 
reasonable,  since  the  configuration/technology 
combinations  for  which  cost-effectiveness 
estimates  are  typically  generated  are  10-15 
years  from  production. 

Multiattribute  Utility  Analysis 

Multiattribute  value  and  utility  methods  are 
receiving  increasing  recognition  and  applica- 
tion in  industry.  At  Lockheed-Georgia , multi- 
attribute methods  have  been  used  to  assist  in 
both  program-level  and  design  decisions.  As 
part  of  the  program  risk  analysis  activity  for 
the  C-X  proposal,  we  built  an  additive  multi- 
attribute utility  model  (Figure  8)  to  represent 
our  customer's  value  system.  This  model  was 
used  to  demonstrate  how  Lockheed's  approach  to 
C-X  risk  areas  minimized  program  risk  (maxi- 
mized expected  utility) . 

Two  design  decision  studies  have  used  multi- 
attribute value  models.  The  first  was  a model 
of  the  C-X  Source  Selection  Criteria,  used  to 
screen  potential  cargo-box  dimension  combina- 
tions. The  second  model  constructed  lends 
structure  to  the  numerous  effectiveness 
criteria  for  a tactical  airlifter.  This  is 
particularly  significant  since  tactical  air- 
lift, with  its  multiplicity  of  missions,  has 


historically  resisted  quantification  of  effec- 
tiveness and  worth. 

To  use  multiattribute  methods  to  solve  a 
management  problem,  the  decision  analyst  needs 
five  things: 

1.  A hierarchy  of  objectives  and  attri- 
butes. 

2.  Characteristics  of  each  alternative  in 
each  of  the  attributes. 

3.  Utility  or  value  functions,  one  for  each 
attribute. 

4.  A weighting  scheme  for  the  hierarchy 

5.  A math  model  which  accepts  the  informa- 
tion in  1,  2,  3,  and  4 and  outputs  util- 
ity scores  for  each  alternative. 


The  analyst  must  devote  significant  time  to 
the  first  four  activities  which  by  their  nature 
require  repeated  iteration  and  significant 
personal  interaction.  For  this  reason,  the 
availability  of  an  automated  model  (item  5) 
becomes  significant.  The  analyst  needs  to  be 
able  to  (1)  input  the  multiattribute  model  and 
the  characteristics  of  the  alternatives  in  the 
attributes,  and  (2)  have  value,  utility,  or 


expected  utility  scores  calculated  and  summed 
rapidly.  A FORTRAN  program  developed  under  R&D 
by  the  project  during  1981  (Reference  3)  ful- 
fills these  two  needs  as  well  as  permitting 
rapid  plotting  of  utility  and  probability 
curves.'  A flow  diagram  is  shown  in  Figure  9. 

The  preliminary  work  of  defining  criteria 
and  alternatives,  as  described  in  Reference  i» , 
is  critical  to  the  success  of  a multiattribute 
decision  analysis.  These  activities  should 
receive  the  majority  of  the  study  time,  with 
numerical  manipulation  on  the  computer  being 
the  final  step  leading  to  recommendation. 

Critical  Path  Method 

The  use  of  CPM  networks  in  conjunction  with 
project  planning  is  well-known.  Our  experience 
in  Advanced  Design  indicates  that  the  diffic- 
ulty in  pinpointing  stal-t  and  end  dates  for 
technology  development  projects  means  that  the 
most  useful  data  a decision  scientist  can  pro- 
vide are;  (1)  how  much  can  a development 
schedule  be  accelerated  (feasibility),  (2) 
which  activities  should  be  accelerated  ( and  in 
what  order),  and  (3)  what  will  each  increment 
cost.  In  the  literature  this  is  known  as  "time/ 
cost  crashing"  and  the  computer  code  we  use  to 
estimate  the  acceleration  is  aptly  named  CPM/ 
CRASH.  A flow  diagram  for  CPM/ CRASH  is  shown 
in  Figure  10. 


Besides  its  basic  usefulness,  this  program 
is  interesting  for  two  reasons.  First,  the 
program  use3  the  algorithm  described  in 
Reference  5,  "A  Flow-Pre3erving  Algorithm  for 
the  Time/Cost  Trade-off  Problem"  and  hence 
represents  a direct  transfer  from  academic 
research  to  industrial  applications.  Second, 
consider  how  CPM/CRASH  works.  Inputs  to  the 
model  ard  (1)  a network  description  of  enabling 
activities,  and  (2)  a linear  time/  co3t  trade- 
off curve  for  each  activity.  CPM/CRASH  system- 
atically accelerates  the  development  project, 
"crashing"  the  activity  on  the  current  critical 
path  which  gives  the  maximum  time  reduction  per 
dollar  until  no  further  acceleration  is  possi- 
ble. Zeleny  (Reference  6,  pp  55-58)  points  out 
that  the  project  time/cost  trade-off  curve  is 
an  "efficient  boundary"  in  the  terminology  of 
multiple  criteria  decisionmaking. 

After  obtaining  a time/cost  curve  for  the 
development  project,  the  decision  analyst  must 
quantify  the  uncertainty  in  the  predicted 
acceleration.  A confidence  band  about  the 
trade-off  curve  is  needed.  We  obtain  such  a 
band  by  developing  a probability  distribution 
on  project  completion  time  at  each  of  several 
costs  ranging  from  normal  to  maximum  accelera- 
tion. We  use  network  simulation  rather  than 
the  PERT  formulas  because  of  their  well-known 
underestimation  of  both  mean  and  variance  in 
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Figure  9,  Flow  Diogrom  for  Multiattribute  Decision  Analysis 
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Figure  10.  Process  to  Determine  Project  Time/Cost  Tradeoff  Curve 


Figure  1 ).  Confidence  Band  on  CPM/CRASH  Results 


PROJECT  STATUS  AND  DIRECTIONS 


In  this  section,  we  sunmarize  where  we  stand 
in  three  areas:  (1)  development  of  a unified 

framework  for  risk  anal>sis  studies  of  DoD 
procurement  programs;  (2)  methods  development 
beyond  the  basic  tools  in  the  preceding  sec- 
tion; and  (3)  applications  to  problems  in  areas 
outside  our  current  environment. 

Program  Risk  Analysis  Approach 

Based  on  two  years  of  methods  development, 
our  current  approach  to  the  types  of  risk 
studies  required  on  new  DoD  business  is  shown 
in  Figure  12.  Our  philosophy  is  that  risk 
emanates  from  the  technical  definition  of  the 
system,  manifesting  itself  in  the  probability 
that  technology,  mission,  cost,  or  schedule 
goals  will  not  be  met.  The  first  two  models 
shown,  QUALM  and  CPM/CRASH,  are  being  used  to 
conduct  the  technology  assessment  task  of  the 
study  contract  "Technology  Alternatives  for 
Airlift  Deployment,"  sponsored  by  USAF  Flight 
Dynamics  Laboratory.  QUALM  is  the  Lockheed- 
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Georgia  software  for  implementing  the  quanti- 
tative uncertainty  analysis  process  depicted 
earlier  in  Figure  5.  The  third  model,  called 
CPM/RISK,  is- an  RAD  task  for  the  project  in 
1983.  This  model  will  take  information  on 
potential  technical  problems,  provided  by  the 
specialties,  and  simulate  the  impact  of  these 
problems  on  a program  activity  network  with 
both  cost  and  schedule  estimates  on  each  arc. 
Model  output  will  be  probability  distributions 
on  project  time  and  cost,  as  well  as  statistics 
for  each  activity  (e.g.,  probability  that  an 
activity  will  be  on  the  project  critical  path). 
CPM/RISK  is  a Loekheed-Georgia  modification  of 
a methodology  conceived  of  by  Kraemer  at 
Boeing-Vertol  (Reference  7). 

Advanced  Tools 


We  are  developing  more  advanced  tools  than 
those  discussed  earlier.  The  reader  can  find 
adequate  references  in  the  open  literature  for 
the  methods  listed  below.  Each  of  these  we 
consider  necessary  to  make  the  step  from  ade- 
quate to  outstanding  methodology  readiness. 


Figure  12.  Uniflmd  Approoch  To  Risk  Analysis 
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Methods  Currently  Reap; 
o Goal  Programing 
o Decision  Trees 
o Statistical  Package 

o Venture  Evaluation  and  Review  Technique 
(VERT) 

o Impact  Diagram  Method 

o Iso-Risk  Contour  Method 

Methods  In  Development  or  Ordered 

o C PM/RISK  Simulation  Model 

o Graphical  Evaluation  and  Review  Technique 
(GERT) 

o Linear  Multiobjective  Programming 
o DYNAMO 

Extended  Project  Environment 

As  described  earlier,  through  1982  the  pro- 
ject applications  had  been  exclusively  to 
activities  within  Lockheed-Georgia's  Engineer- 
ing Branch.  Through  a series  of  briefings  in 


early  1983,  we  have  reached  out  to  other 
branches  of  the  company,  to  other  divisions  of 
the  corporation,  and  to  potential  DoD  custo- 
mers . The  project  environment  has  therefore 
been  extended  as  shown  in  Figure  13,  resulting 
in  a respectable  demand  for  our  services.  For 
example,  we  are  currently  engaged  in  studies 
for  two  directorates  which  report  directly  to 
the  president  . — Strategic  Planning  and 
Advanced  Programs.  He  also  are  negotiating  a 
government-funded  study  contract  for  1983-8*1. 
These  requests  for  our  services  simply  reflect 
the  fact  that  managers  in  technologically- 
oriented  work  have  problems  for  which  the 
decision  analyst  can  offer  structuring  and 
quantitative  insight. 

While  methods  development  will  continue  in 
198*1,  we  see  our  division  of  effort  shifting 
from  half-time  methods  development  and  half- 
time applications  to  65-70J  applications.  In 
our  view,  we  have  tremendous  advantages  over  an 
outside  consultant  when  working  on  Lockheed- 
Georgia  problems,  even  outside  of  our  immediate 
environment  — we  are  trusted  with  proprietary 
data,  we  are  aware  of  problem  subtleties  and 
personalities,  and  we  know  the  organization 
chain,  both  formal  and  informal.  In  summary, 
we  are  becoming  the  "in-house  decision  analysis 
staff"  at  Lockheed-Georgia , a development  which 
Ulvila  and  Brown  forecast  for  all  large  corpo- 
rations in  their  recent  paper  "Decision 
Analysis  Comes  of  Age,"  Reference  8. 


Figure  13.  Extended  Project  Environment 
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CONCLUSIONS 


REFERENCES 


The  need  for  quantitative  risk  assessment  on 
DoD  technology  studies  and  procurement  programs 
has  been  recognized  since  the  late  60s. 
Methodology  evolved  during  the  70s  to  meet  this 
need,  and  is  now  readily  available.  Methods 
exist  to  assess  uncertainty  in  developing 
technologies  — uncertainties  in  time  and  cost 
to  reach  maturity,  and  uncertainties  in  opera- 
tional benefit  at  maturity.  Once  technologies 
advance  to  a point  were  they  may  be  proposed  on 
a new  aircraft  programs,  methods  exist  to  help 
identify  and  quantify  program  risk. 

An  analyst  equipped  with  methods  as  describ- 
ed in  this  paper  can  contribute  to  the  system/ 
program  definition  and  analysis.  This  contri- 
bution, of  course,  is  dependent  on  management 
acceptance  and  engineering  specialty  support. 
Management  wants  risk  studies  performed  because 
they  are  acutely  aware  of  uncertainties,  and 
because  their  counterparts  in  the  government 
require  risk  be  identified  and  measured. 
Engineers  will  cooperate  with  the  risk  analyst 
once  they  have  seen  that  their  specialty/judg- 
ment will  not  be  misrepresented  The  risk 
analyst  has  become  an  accepted  member  of  con- 
ceptual design  teams  at  Lockheed-Georgia,  much 
as  the  cost  analyst  did  10-15  years  earlier. 

Risk  analysis  methods  are  treated  as  a 
technology  at  Lockheed-Georgia.  IR&D  funding 
permits  analysts  to  improve  existing  methods, 
to  perform  research,  and  to  create  new  methods 
and  the  accompanying  computer  codes.  This 
funding  has  permitted  us  to  build  an  adequate 
framework  for  risk  analysis  of  defense  systems 
and  programs  in  less  than  three  years.  He  have 
expanded  into  related  decision  sciences  of 
network,  analysis,  statistical  analysis,  and 
decision  analysis.  While  continuing  to  provide 
risk  studies  in  engineering  as  our  primary 
duty,  we  are  now  working  on  a broader  variety 
of  problems  throughout  the  company. 


1.  Air  Force  Systems  Command,  Aeronautical 

Systems  Division,  Request  for  Proposal  No. 
F33657-80-R-0227,  "Full  Scale  Engineering 
Development/Production:  Interim  Contrac- 

tor Support  and  Spares  for  the  C-X  Air- 
craft System,"  October  1980. 

2.  Batson,  R.  G.,  "Description  of  the  Model 

AFCUE:  Airlift  Fleet  Cost-Effectiveness 

Uncertainty  Estimator  for  Parametric 
Design  Studies,"  Lockheed-Georgia  Company, 
PM92D36H-6,  August  1982. 

3.  Bailey,  T.  M. , and  R.  G.  Batson,  "Multi- 

attribute Decision  Analysis  Model  (MADAM): 
Program  Description  and  User's  Guide," 
Lockheed-Georgia  Company,  LG81WP7292-002, 
August  1981. 

9.  Keeney,  R.  L. , and  H.  Raiffa,  Decisions  with 
Multiple  Objectives:  Preferences  and 

Value  Tradeoffs,  John  Wiley  & Sons,  New 
York,  1976. 

5.  Tufekci,  S. , "A  Flow-Preserving  Algorithm 

for  the  Time/Cost  Trade-off  Problem,"  HE 
Transactions,  Vol.  19,  No.  2,  June  1982, 
pp.  109-113. 

6.  Zeleny,  M.,  Multiple  Criteria  Decision 

Making,  McGraw-Hill,  New  York,  1982. 

7.  Kraemer,  G.  T. , "A  Successful  Quantitative 

Risk  Assessment  Technique,"  Journal  of 
Parametrlcs,  Vol.  1,  No.  3.  Winter  1988, 
pp.  13-21. 

8.  Ulvila,  J.  W. , and  R.  V.  Brown,  "Decision 

Analysis  Comes  of  Age,"  Harvard  Business 
Review,  Sep-Oct  1982,  pp.  130-191. 


